Display system, display device, information display method, and program

ABSTRACT

A display system includes a first display device of a head mounted type, and a second display device. The first display device includes: a first display configured to display a guide image, a first camera configured to capture an outside image, and a first communication interface configured to: transmit the outside image to the second display device, and receive the guide image transmitted from the second display device. The second display device includes: a second display configured to display the outside image, a detector configured to detect a pointing position relative to the outside image, a second processor configured to generate the guide image based on the pointing position, and a second communication interface configured to: receive the outside image transmitted from the first display device, and transmit the guide image to the first display device.

This is a Continuation of U.S. patent application Ser. No. 14/963,962filed Dec. 9, 2015. The entire content of the prior application isincorporated by reference herein in its entirety.

BACKGROUND 1. Technical Field

The present invention relates to a display system, a display device, aninformation display method, and a program.

2. Related Art

In the past, there has been known an example of using a display devicemounted on the head and called a head mounted display (HMD) in order tosupport a person performing a work (see, e.g., JP-A-2007-320715(Document 1) and JP-A-2012-138005 (Document 2)). The system described inDocument 1 transmits a view field image, which has been taken by acamera device provided to a worker HMD unit, to a manager HMD unit tomake it possible to monitor the working state of the worker in themanager HMD unit. Further, the server described in Document 2 transmitsinformation related to a destination, goods, schedule, and so on to anHMD worn by a pilot of a transport aircraft. A client device providedwith the HMD displays virtual space information on the HMD based on theinformation transmitted from the server, position information andorientation information of the transport aircraft, and so on. Thevirtual space information includes a variety of types of informationexpressed by arrows or characters.

Although in the related art method, the view field image and theinformation using characters are displayed in the HMD on the side ofsupporting the work or on the side on which the work is supported asdescribed above, it has been desired to transmit a larger amount ofinformation in a way easier to understand.

SUMMARY

An advantage of some aspects of the invention is to make it possible totransmit a larger amount of information in a way easier to understand toa person wearing a head-mounted display device.

A display system according to an aspect of the invention includes afirst display device of a head mounted type, and a second display deviceof the head mounted type, the first display device includes a firstdisplay section adapted to display an image so that an outside view canvisually be recognized, an imaging section adapted to take an image of arange including at least a part of the outside view, which can visuallybe recognized in the first display section, a first communicationsection adapted to transmit guide target information generated based ona taken image of the imaging section to the second display device, andreceive guide information from the second display device, and a firstcontrol section adapted to make the first display section display animage based on the guide information received in the first communicationsection, and the second display device includes a second display sectionadapted to display an image so that the outside view can visually berecognized, a second communication section adapted to receive the guidetarget information from the first display device, and transmit the guideinformation to the first display device, a display control sectionadapted to make the second display section display an image based on theguide target information received by the second communication section,an operation detection section adapted to detect an operation, and asecond control section adapted to generate the guide informationincluding information representing an operation detected by theoperation detection section during a period of displaying an image basedon the guide target information, information adapted to associate anoperation and the guide target information with each other, and transmitthe guide information using the second communication section.

According to the aspect of the invention, the guide target informationbased on the taken image taken by the first display device istransmitted to the second display device, the second display devicedisplays the image based on the guide target information, and the guideinformation including the information for associating the operationduring the period of displaying the image and the guide targetinformation with each other is transmitted to the first display device.Therefore, since the information related to the operation associatedwith the taken image in the first display device can be provided to thefirst display device using the operation of the second display device,it is possible to transmit a large amount of information to the user ofthe first display device in an easy-to-understand manner.

According to another aspect of the invention, in the display systemdescribed above, the first control section provided to the first displaydevice generates the guide target information including the taken imageof the imaging section, the operation detection section provided to thesecond display device detects a position pointing operation, and thesecond control section generates the guide information includinginformation adapted to associate a pointing position of the positionpointing operation detected by the operation detection section with thetaken image included in the guide target information.

According to this aspect of the invention, the position pointingoperation can be performed in the second display device, and in thefirst display device, it is possible to make the pointing position ofthe position pointing operation in the second display device correspondto the taken image. Therefore, it is possible to transmit the content ofthe position pointing operation performed in the second display deviceto the user of the first display device.

According to another aspect of the invention, in the display systemdescribed above, the second control section provided to the seconddisplay device generates the guide information including informationrelated to the pointing position of the position pointing operationdetected by the operation detection section, and an image of a pointingobject, and the first control section provided to the first displaydevice makes the first display section display the image of the pointingobject at a position corresponding to the pointing position included inthe guide information received by the first communication section.

According to this aspect of the invention, by performing the positionpointing operation in the second display device, the first displaydevice displays the image of the pointing object in accordance with thepointing position by the position pointing operation. Therefore, due tothe operation of the second display device, it is possible to point theposition to the user of the first display device.

According to another aspect of the invention, in the display systemdescribed above, the first control section provided to the first displaydevice makes the first display section display an image for a GUI(graphical user interface) operation, and detects an operation based onthe display position of the image for the GUI operation and the pointingposition included in the guide information received by the secondcommunication section.

According to this aspect of the invention, by performing the positionpointing operation in the second display device, it is possible toperform the GUI operation to the first display device to control thefirst display device.

According to another aspect of the invention, in the display systemdescribed above, the first control section provided to the first displaydevice detects an operation to the first display device based oninformation representing an operation included in the guide informationreceived by the first communication section.

According to this aspect of the invention, by performing the operationin the second display device, it is possible to control the firstdisplay device.

According to another aspect of the invention, in the display systemdescribed above, in a case in which the guide target informationreceived by the second communication section includes the taken image,the second control section provided to the second display device makesthe second display section display the taken image, edits the takenimage based on the operation detected by the operation detection sectionduring a period of displaying the taken image, and generates the guideinformation including the taken image edited.

According to this aspect of the invention, it is possible to edit thetaken image taken in the first display device to show the result to theuser of the first display device.

According to another aspect of the invention, in the display systemdescribed above, the second display device transmits information adaptedto designate imaging condition of the imaging section provided to thefirst display device using the second communication section, and thefirst control section of the first display device receives the imagingcondition from the second display device, controls the imaging sectionbased on the imaging condition received, and transmits the guide targetinformation generated based on the taken image of the imaging section tothe second display device.

According to this aspect of the invention, by designating the imagingcondition from the second display device, the guide target informationbased on the taken image taken in the imaging condition thus designatedcan be obtained in the second display device. Therefore, the user of thesecond display device can obtain the desired information in theenvironment of the first display device.

According to another aspect of the invention, in the display systemdescribed above, the second display device transmits information adaptedto designate one of an imaging direction, a zoom factor, and the imagingdirection and the zoom factor as the imaging condition using the secondcommunication section, and the first control section of the firstdisplay device controls one of the imaging direction, the zoom factor,and the imaging direction and the zoom factor based on the imagingcondition received from the second display device. According to thisaspect of the invention, by designating the imaging direction or thezoom factor from the second display device, the guide target informationbased on the taken image taken in the imaging condition thus designatedcan be obtained in the second display device. Therefore, the user of thesecond display device can obtain the information related to the desiredtaken image in the environment of the first display device.

A head mounted display according to still another aspect of theinvention includes a display section adapted to display an image so thatan outside view can visually be recognized, an imaging section adaptedto take an image of a range including at least a part of the outsideview, which can visually be recognized in the display section, acommunication section adapted to transmit guide target informationgenerated based on a taken image of the imaging section to an externaldevice, and receive guide information from the external device, and acontrol section adapted to calculate a display position of an imagebased on the guide information received by the communication section,and make the display section display an image based on the guideinformation at a display position calculated.

According to this aspect of the invention, the guide target informationbased on the taken image in a range including a part of the outside viewis transmitted to the external device, and the image is displayed inaccordance with the guide information received from the external device.Thus, it is possible to transmit the information related to theoperation presently performed to the external device, and to efficientlyshow the information transmitted by the external device to the user.

An information display method according to yet another aspect of theinvention includes providing a first display device including a firstdisplay section adapted to display an image so that an outside view canvisually be recognized, and a second display device including a seconddisplay section adapted to display an image so that an outside view canvisually be recognized, imaging a range including at least a part of theoutside view which can visually be recognized in the first displaysection using the first display device, and transmitting guide targetinformation generated based on the taken image to the second displaydevice, receiving the guide target information from the first displaydevice using the second display device, making the second displaysection display the image based on the guide target informationreceived, detecting an operation during a period of displaying the imagebased on the guide target information, generating guide informationincluding information representing the operation detected, andinformation adapted to associate the detected operation and the guidetarget information with each other, and transmitting the guideinformation using the second communication section, and receiving theguide information transmitted by the second display device using thefirst display device, and making the first display section display theimage based on the guide information received.

According to this aspect of the invention, the guide target informationbased on the taken image taken by the first display device istransmitted to the second display device, the second display devicedisplays the image based on the guide target information, and the guideinformation including the information for associating the operationduring the period of displaying the image and the guide targetinformation with each other is transmitted to the first display device.Therefore, since the information related to the operation associatedwith the taken image in the first display device can be provided to thefirst display device using the operation of the second display device,it is possible to transmit a large amount of information to the user ofthe first display device in an easy-to-understand manner.

A program according to still yet another aspect of the invention isexecuted by a computer adapted to control a head mounted displayprovided with a display section adapted to display an image so that anoutside view can visually be recognized and an imaging section adaptedto take an image of a range including at least a part of the outsideview which can visually be recognized in the display section, theprogram including transmitting guide target information generated basedon a taken image of the imaging section to an external device using thecomputer, receiving guide information from the external device,calculating a display position of the image based on the guideinformation received, making the display section display the image basedon the guide information at a display position calculated.

According to this aspect of the invention, the guide target informationbased on the taken image in a range including a part of the outside viewis transmitted to the external device, and the image is displayed inaccordance with the guide information received from the external deviceusing a display device. Therefore, it is possible to transmit theinformation related to the operation presently performed to the externaldevice, and to efficiently show the information transmitted by theexternal device to the user.

As further another aspect of the invention, the invention can beconfigured as a computer readable recording medium storing the program.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be described with reference to the accompanyingdrawings, wherein like numbers reference like elements.

FIG. 1 is a general configuration diagram of a display system accordingto a first embodiment of the invention.

FIG. 2 is an explanatory diagram showing an exterior configuration of ahead mounted display.

FIG. 3 is a diagram showing a configuration of an optical system of animage display section.

FIG. 4 is a functional block diagram of components constituting the headmounted display in a command site.

FIG. 5 is a functional block diagram of components constituting the headmounted display in a working site.

FIGS. 6A and 6B are flowcharts showing an action of a display system.

FIGS. 7A through 7D are diagrams each showing a display example of ahead mounted display.

FIGS. 8A and 8B are flowcharts showing an action of a display systemaccording to a second embodiment of the invention.

FIG. 9 is a flowchart showing an action of a display system according tothe second embodiment.

FIGS. 10A through 10D are diagrams each showing a display example of ahead mounted display according to the second embodiment.

FIGS. 11A and 11B are flowcharts showing an action of a display systemaccording to a third embodiment of the invention.

FIGS. 12A and 12B are flowcharts showing an action of a display systemaccording to a fourth embodiment of the invention.

DESCRIPTION OF EXEMPLARY EMBODIMENTS First Embodiment

FIG. 1 is a schematic configuration diagram of a display system 1according to an embodiment to which the invention is applied. As shownin FIG. 1, the display system 1 is a system obtained by connecting aplurality of HMD 100 to each other via a communication network 4 so asto be able to communicate with each other. Although FIG. 1 shows anexample of connecting the three HMD 100 to the communication network 4,the number of the HMD 100 provided to the display system 1 is notparticularly limited.

In the present embodiment, the display system 1 is provided with twotypes of HMD 100 having respective functions different from each other,and the two types of HMD 100 are respectively referred to as HMD 100Aand HMD 100B. In the case in which it is not necessary to separate HMD100A and HMD 100B from each other, the description of HMD 100 is used.

The HMD 100A, 100B are each a display device worn by the user on thehead as shown in FIG. 1, and are also called a head mounted display. TheHMD 100A, 100B are each an optical transmissive HMD allowing the user tovisually recognize a virtual image and at the same time visuallyrecognize an external sight directly.

It should be noted that in the following explanation, the virtual imageto be visually recognized by the user using the HMD 100A, 100B is alsoreferred to as a “display image” for the sake of convenience. Further,emission of the image light generated based on image data is alsoreferred to as “display of the image.”

In the display system 1, a worker performing a work wears the HMD 100B,a commander for making an instruction to or performing management of theworker wears the HMD 100A, and information for the instruction and themanagement is transmitted and received between the commander and theworker. There is no geographical constraint between a command site Awhere the commander is located and a working site B where the workerperforms the work providing the connection to the communication network4 is available, and the command site A and the working site B can bedistant from each other, or can also be close to each other.

In the display system 1, the HMD 100B worn by the worker has a functionof taking an image of the view field of the worker and then transmittingthe work information (guide object information) including the takenimage to the HMD 100A. The HMD 100A receives the work information, andthen displays the taken image of the view field of the worker. Thecommander operates the HMD 100A to perform a selection of an explanatorycontent, an action of moving a pointing object (e.g., an arm, a hand, ora finger) for an explanation, and so on, and thus, instructioninformation (guide information) including the data representing thecontent thus selected and the movement of the pointing object istransmitted from the HMD 100A to the HMD 100B. The HMD 100A is aso-called see-through display device with which the outside view canvisually be recognized. Therefore, the commander performs the operationwhile observing the view field of the worker displayed by the HMD 100A.

The HMD 100B receives the instruction information transmitted by the HMD100A to display the image of the content and the image showing themotion of the pointing object on an image display section 20. Since theHMD 100B is a see-through display device, it is possible for the workerto perform a work looking at the hands while viewing the image displayedby the image display section 20.

Thus, in the display system 1, it is possible for the commander locatedin the command site A to visually recognize the view field of the workerperforming a work in the working site B, and then provide a guide and anexplanation of the work to the HMD 100B of the worker using the pointingobject and the content. It is possible for the worker to perform thework while watching the image of the content and the motion of thepointing object. The communication network 4 is realized by a variety oftypes of communication lines such as a public line network, a dedicatedline, a wireless communication line including a mobile phone line, andbackbone communication lines of these lines, or a combination of any ofthese lines, and the specific configuration thereof is not particularlylimited. The communication network 4 can be a wide area communicationline network capable of connecting remote locations to each other, orcan also be a local area network (LAN) set in a specific facility orbuilding. Further, the communication network 4 can also include networkequipment such as server devices, gateway devices, and router devicesfor connecting the variety of types of communication lines describedabove to each other. Further, the communication network 4 can also beformed of a plurality of communication lines. In the command site A,there is disposed an access point 3A for providing connection to thecommunication network 4, and in the working site B, there is disposed anaccess point 3B for providing connection to the communication network 4.The wireless access points 3A, 3B are each a communication device suchas an access point or a router, and relay the data communication betweenthe HMD 100A, 100B and the communication network 4, respectively.

The HMD 100A executes data communication with other HMD 100B via thewireless access points 3. It should be noted that it is also possiblefor each of the HMD 100A to execute the wireless communication withother HMD 100B directly in, for example, an ad hoc mode, or it is alsopossible to connect the HMD 100A, 100B to each other using a wiredcommunication line. In other words, the configuration of the displaysystem 1 is not particularly limited providing the HMD 100A and the HMD100B are capable of communicating with each other.

The HMD 100A is provided with an image display section 20 worn by thecommander as a user on the head, and a control device 10A forcontrolling the image display section 20. The image display section 20allows the user to visually recognize a virtual image in a state ofbeing worn on the head of the user. The control device 10A alsofunctions as a controller for the user to operate the HMD 100A.

Further, the HMD 100B is provided with the image display section 20 anda control device 10B. The image display section 20 is common to the HMD100A, 100B, and allows the user to visually recognize a virtual image ina state of being worn on the head of the user. The control device 10Bcontrols the image display section 20 similarly to the control device10A, and also functions as a controller for the user to operate the HMD100B. The control device 10A and the control device 10B have a commonexterior appearance, but differ in a functional configuration from eachother. The difference will be described later. FIG. 2 is an explanatorydiagram showing an exterior configuration of the HMD 100A. Since the HMD100B is the same in the exterior appearance as the HMD 100A, thegraphical description and the explanation of the configuration of theHMD 100B will be omitted.

The image display section 20 is a mount body to be mounted on the headof the user, and has a shape of a pair of glasses in the presentembodiment. The image display section 20 is provided with a rightholding section 21, a right display drive section 22, a left holdingsection 23, a left display drive section 24, a right optical imagedisplay section 26, a left optical image display section 28, a camera61, and a microphone 63. The right optical image display section 26 andthe left optical image display section 28 are disposed so as to belocated in front of the right and left eyes of the user, respectively,when the user wears the image display section 20. One end of the rightoptical image display section 26 and one end of the left optical imagedisplay section 28 are connected to each other at a positioncorresponding to the glabella of the user when the user wears the imagedisplay section 20.

The right holding section 21 is a member disposed so as to extend froman end portion ER, which is the other end of the right optical imagedisplay section 26, to a temporal region of the head of the user whenthe user wears the image display section 20. Similarly, the left holdingsection 23 is a member disposed so as to extend from an end portion EL,which is the other end of the left optical image display section 28, toa temporal region of the head of the user when the user wears the imagedisplay section 20. The right holding section 21 and the left holdingsection 23 hold the image display section 20 on the head of the user insuch a manner as the temples of the pair of glasses.

The right display drive section 22 and the left display drive section 24are disposed on the sides to be opposed to the head of the user when theuser wears the image display section 20. It should be noted that theright display drive section 22 and the left display drive section 24 arealso collectively referred to simply as a “display drive section,” andthe right optical image display section 26 and the left optical imagedisplay section 28 are also collectively referred to simply as an“optical image display section.”

The display drive sections 22, 24 respectively include liquid crystaldisplays 241, 242 (hereinafter also referred to as “LCD 241, 242”),projection optical systems 251, 252, and so on (see FIG. 4). The detailsof the configuration of the display drive sections 22, 24 will bedescribed later. The optical image display sections 26, 28 as opticalmembers are respectively provided with light guide plates 261, 262 (seeFIG. 4), and dimming plates 20A. The light guide plates 261, 262 areeach formed of light transmissive resin or the like, and guide the imagelight output from the display drive sections 22, 24 to the eyes of theuser, respectively. Each of the dimming plates 20A is a thin-plate likeoptical element, and is disposed so as to cover the obverse side of theimage display section 20, which is the side opposite to the side of theeyes of the user. As the dimming plates 20A, there are used a variety ofmembers such as a member with nearly zero light permeability, a nearlytransparent member, a member for transmitting light while reducing thelight intensity, or a member for attenuating or reflecting the lightwith a specific wavelength. By appropriately selecting the opticalcharacteristics (e.g., the light transmittance) of the dimming plates20A, it is possible to control the intensity of the outside lightexternally entering the right optical image display section 26 and theleft optical image display section 28 to thereby control easiness of thevisual recognition of the virtual image. In the description of thepresent embodiment, there is explained the case of using the dimmingplates 20A having at least such light transmittance that the userwearing the HMD 100 can visually recognize the outside scenery. Thedimming plates 20A protect the right light guide plate 261 and the leftlight guide plate 262 to suppress damages, adhesion of dirt, and so onto the right light guide plate 261 and the left light guide plate 262,respectively.

The dimming plates 20A can be arranged to be detachably attached to theright optical image display section 26 and the left optical imagedisplay section 28, or it is possible to arrange that a plurality oftypes of dimming plates 20A can be attached while being replaced witheach other, or it is also possible to eliminate the dimming plates 20A.

The camera 61 is disposed in a boundary portion between the rightoptical image display section 26 and the left optical image displaysection 28. In the state in which the user wears the image displaysection 20, the position of the camera 61 is roughly the middle of theboth eyes of the user in a horizontal direction, and is above the botheyes of the user in the vertical direction. The camera 61 is a digitalcamera provided with an imaging element such as CCD or CMOS, an imaginglens, and so on, and can also be a monocular camera or a stereo camera.

The camera 61 takes an image of at least a part of the outside view inthe obverse side direction of the HMD 100, in other words, in the viewfield direction of the user in the state of wearing the HMD 100A.Although the width of the field angle of the camera 61 can arbitrarilybe set, it is preferable that the imaging range of the camera 61corresponds to the range including the external sight visuallyrecognized by the user through the right optical image display section26 and the left optical image display section 28. Further, it is morepreferable that the imaging range of the camera 61 is set so that theimage of the entire view field of the user through the dimming plates20A can be taken. The camera 61 takes the image in accordance with thecontrol by the control section 140, and then outputs the taken imagedata to the control section 140.

FIG. 3 is a plan view of a substantial part showing a configuration ofan optical system provided to the image display section 20. FIG. 3 showsthe left eye LE and the right eye RE of the user for the sake ofexplanation.

The left display drive section 24 is provided with a left backlight 222having a light source such as LED and a diffusion plate, a left LCD 242of a transmissive type disposed on the light path of the light emittedfrom the diffusion plate of the left backlight 222, and a leftprojection optical system 252 including a lens group for guiding theimage light L having passed through the left LCD 242 and so on. The leftLCD 242 is a transmissive liquid crystal panel having a plurality ofpixels arranged in a matrix.

The left projection optical system. 252 includes a collimating lens forconverting the image light L having been emitted from the left LCD 242into a light beam in a parallel state. The image light L converted bythe collimating lens into the light beam in the parallel state entersthe left light guide plate 262. The left light guide plate 262 is aprism provided with a plurality of reflecting surfaces for reflectingthe image light L, and the image light L is guided to the left eye LEthrough a plurality of times of reflection inside the left light guideplate 262. The left light guide plate 262 is provided with a half mirror262A (a reflecting surface) located in front of the left eye LE.

The image light L having been reflected by the half mirror 262A isemitted from the left optical image display section 28 toward the lefteye LE, and then the image light L forms an image on the retina of theleft eye LE to make the user visually recognize the image.

The right display drive section 22 is formed so as to be bilaterallysymmetric with the left display drive section 24. The right displaydrive section 22 is provided with a right backlight 221 having a lightsource such as LED and a diffusion plate, a right LCD 241 of atransmissive type disposed on the light path of the light emitted fromthe diffusion plate of the right backlight 221, and a right projectionoptical system 251 including a lens group for guiding the image light Lhaving passed through the right LCD 241 and so on. The right LCD 241 isa transmissive liquid crystal panel having a plurality of pixelsarranged in a matrix.

The right projection optical system 251 includes a collimating lens forconverting the image light L having been emitted from the right LCD 241into a light beam in a parallel state. The image light L converted bythe collimating lens into the light beam in the parallel state entersthe right light guide plate 261. The right light guide plate 261 is aprism provided with a plurality of reflecting surfaces for reflectingthe image light L, and the image light L is guided to the right eye REthrough a plurality of times of reflection inside the right light guideplate 261. The right light guide plate 261 is provided with a halfmirror 261A (a reflecting surface) located in front of the right eye RE.

The image light L having been reflected by the half mirror 261A isemitted from the right optical image display section 26 toward the righteye RE, and then the image light L forms an image on the retina of theright eye RE to make the user visually recognize the image.

The image light L having been reflected by the half mirror 261A andoutside light OL having passed through the dimming plate 20A enter theright eye RE of the user. The image light L having been reflected by thehalf mirror 262A and the outside light OL having passed through thedimming plate 20A enter the left eye LE. As described above, the HMD 100makes the image light L having been processed inside and the outsidelight OL overlap each other and then enter the eyes of the user, and itis possible for the user to see the outside view through the dimmingplate 20A, and to visually recognize the image due to the image light Lso as to be superimposed on the outside view. As described above, theHMD 100 functions as a see-through display device.

It should be noted that the left projection optical system. 252 and theleft light guide plate 262 are also collectively referred to as a “leftlight guide section,” and the right projection optical system 251 andthe right light guide plate 261 are also collectively referred to as a“right light guide section.” The configuration of the right light guidesection and the left light guide section is not limited to the exampledescribed above, but an arbitrary method can be used as long as thevirtual image is formed in front of the eyes of the user using the imagelight, and for example, it is also possible to use a diffractiongrating, or to use a semi-transmissive reflecting film.

The image display section 20 is connected to the control device 10A viaa connection section 40. The connection section 40 is provided with amain body cord 48 to be connected to the control device 10A, a rightcord 42, a left cord 44, and a coupling member 46. The main body cord 48is branched into two cords to form the right cord 42 and the left cord44. The right cord 42 is inserted into the housing of the right holdingsection 21 from a tip portion AP in the extending direction of the rightholding section 21, and is connected to the right display drive section22. Similarly, the left cord 44 is inserted into the housing of the leftholding section 23 from a tip portion AP in the extending direction ofthe left holding section 23, and is connected to the left display drivesection 24.

The coupling member 46 is disposed at a branch point of the main bodycord 48, and the right cord 42 and the left cord 44, and has a jack towhich an earphone plug 30 is connected. A right earphone 32 and a leftearphone 34 extend from the earphone plug 30. A microphone 63 isdisposed in the vicinity of the earphone plug 30. The wiring from theearphone plug 30 to the microphone 63 is bundled as a single cord, andthe cord is branched at the microphone 63 to be connected respectivelyto the right earphone 32 and the left earphone 34.

As shown in, for example, FIG. 1, the microphone 63 is disposed so thata sound collection section of the microphone 63 faces to the eyedirection of the user, and collects the sound to output a sound signalto a sound processing section 187 (FIG. 4). The microphone 63 can be,for example, a monaural microphone or a stereo microphone, or can alsobe a directional microphone, or an omnidirectional microphone.

The right cord 42, the left cord 44, and the main body cord 48 are onlyrequired to be capable of transmitting digital data, and each can beformed of, for example, a metal cable or an optical fiber. Further, itis also possible to adopt a configuration of bundling the right cord 42and the left cord 44 into a single cord.

The image display section 20 and the control device 10A transmit avariety of signals via the connection section 40. There are providedconnectors (not shown) to be fitted with each other respectively to anend portion of the main body cord 48 on the opposite side to thecoupling member 46, and the control device 10A. By fitting the connectorof the main body cord 48 and the connector of the control device 10A toeach other, or releasing the fitting, it is possible to connect ordisconnect the control device 10A and the image display section 20 to orfrom each other.

The control device 10A controls the HMD 100. The control device 10A isprovided with switches including a determination key 11, a lightingsection 12, a display switching key 13, a luminance switching key 15,arrow keys 16, a menu key 17, and a power switch 18. Further, thecontrol device 10A is provided with a track pad 14 operated by the userwith the hand and the fingers.

The determination key 11 detects a holding-down operation, and thenoutputs a signal for determining the content of the operation in thecontrol device 10A. The lighting section 12 is provided with a lightsource such as light emitting diodes (LED) to make a notification of theoperation state (e.g., an ON/OFF state of the power) of the HMD 100. Thedisplay switching key 13 outputs, for example, a signal for instructingswitching of the display mode of the image in accordance with aholding-down operation.

The track pad 14 has an operation surface for detecting a touchoperation, and outputs an operation signal in accordance with theoperation to the operation surface. The detection method in theoperation surface is not particularly limited, but there can be adoptedan electrostatic method, a pressure detection method, an optical method,and so on. The luminance switching key 15 outputs a signal forinstructing an increase or a decrease of the luminance of the imagedisplay section 20 in accordance with a holding-down operation. Thearrow keys 16 outputs an operation signal in accordance with aholding-down operation to the keys corresponding to up, down, right, andleft directions. The power switch 18 is a switch for switching betweenON and OFF of the power of the HMD 100.

Regarding the content explained with reference to FIG. 2, theconfiguration is common to the HMD 100B and the HMD 100A. In otherwords, the configuration of the HMD 100B is obtained by replacing thecontrol device 10A with the control device 10B in the above explanationof the configuration.

FIG. 4 is a functional block diagram of the components constituting theHMD 100A. Further, FIG. 5 is a functional block diagram of thecomponents constituting the HMD 100B. As shown in FIG. 4, the HMD 100Ais provided with an interface 114 to which a variety of types ofexternal equipment OA to be a supply source of the content areconnected. As the interface 114, there can be used an interfacecompatible with wired connection such as a USB interface, a micro USBinterface, or an interface for a memory card, and it is also possiblefor the interface 114 to be formed of a wireless communicationinterface. The external equipment OA is an image supply device forsupplying the HMD 100 with an image, and there are used a personalcomputer (PC), a cellular phone unit, a portable game machine, and soon.

The control device 10A includes a control section 140, an inputinformation acquisition section 110, a storage section 120, atransmitting section (Tx) 51, and a transmitting section (Tx) 52.

The input information acquisition section 110 is connected to anoperation section 135 including the track pad 14, the arrow keys 16, thepower switch 18, and so on described above. The input informationacquisition section 110 detects the operation content in the operationsection 135 based on the signal input from the operation section 135,and then outputs the operation data representing the operation contentto the control section 140. Further, the control device 10A is providedwith a power supply section (not shown), and supplies each section ofthe control device 10A and the image display section 20 with the power.

The storage section 120 is a nonvolatile storage device, and stores avariety of computer programs to be executed by the control section 140,and the data related to these programs. Further, it is also possible forthe storage section 120 to store the data of a still image and a movingimage to be displayed on the image display section 20.

The storage section 120 stores configuration data 121. The configurationdata 121 includes a variety of types of setting values used by thecontrol section 140. The setting values included in the configurationdata 121 can be values input by the operation of the operation section135 in advance, or it is also possible to receive the setting valuesfrom the external equipment OA or other devices (not shown) via thecommunication section 117 or the interface 114, and then store thesetting values.

Further, the storage section 120 stores detecting data 122, menu imagedata 123, transmission image data 124, and instruction content data 125.

The detecting data 122 is the data used for detecting an image of a workobject from the image data. The work object is a target of the workperformed by the worker wearing the HMD 100B, and it is conceivable thatthe work object is located in the view field of the worker when working.The detecting data 122 includes, for example, the data used for theprocess of extracting the image of the work object from the image data,and more specifically includes the data of the feature amount of theimage of the work object. The feature amount can also be the featureamount of the color or the feature amount of the shape, and it is alsopossible to include a plurality of feature amounts in accordance withthe number of colors and the resolution of the image data as theprocessing target. In the display system 1, in the case in which thework object targeted by the worker wearing the HMD 100B has previouslybeen found out, the detecting data 122 related to the work object isstored in the storage section 120.

The menu image data 123 is the data of the image for displaying the menuimage on the image display section 20. The menu image is, for example, amenu bar having icons for the operations arranged in a line, or adial-type menu having icons arranged in a circular frame.

The transmission image data 124 is the image data used for the pointinginformation transmitted by the HMD 100A to the HMD 100B, and is theimage data of an image of the pointing object such as an arm, a hand, afinger, or a pointer rod. Although the details will be described later,it is possible to include a taken image of the pointing object taken bythe camera 61 in the pointing information, but it is also possible touse the image of the pointing object prepared in advance instead of thetaken image. The transmission image data 124 is the image data which canbe included in the pointing information instead of the taken image. Thetransmission image data 124 is, for example, the image data of an imageresembling an arm, a hand, or a finger of the worker, but can also bethe image data of an image of a pointer constituted by a symbol such asan arrow, and the type or the content of the transmission image data 124is not particularly limited.

The instruction content data 125 is the data of the content transmittedby the HMD 100A to the HMD 100B, and can include text data, still imagedata, moving image data, sound data, and so on.

To the control section 140, there are connected a triaxial sensor 113, aGPS 115, and a communication section 117. The triaxial sensor 113 is atriaxial acceleration sensor, and the control section 140 obtains thedetection value of the triaxial sensor 113. Due to the triaxial sensor113, the control section 140 is capable of detecting the motion of thecontrol device 10A, and is capable of detecting an operation of, forexample, swinging the control device 10A. Further, the triaxial sensor113 can also be replaced with a nine-axis sensor. In this case, it ispossible for the control section 140 to obtain the detection values of atriaxial acceleration sensor, a triaxial angular velocity sensor, and atriaxial geomagnetic sensor to thereby detect the attitude, theorientation, and the motion of the control device 10A.

The GPS 115 is provided with an antenna (not shown), and receives a GPS(Global Positioning System) signal to calculate the present position ofthe control device 10A. The GPS 115 outputs the present position and thecurrent time, which have been obtained based on the GPS signal, to thecontrol section 140. Further, it is also possible for the GPS 115 to beprovided with a function of obtaining the current time based on theinformation included in the GPS signal to correct the time kept by thecontrol section 140.

The communication section 117 performs wireless data communicationcompliant with a standard such as the wireless LAN (WiFi (registeredtrademark)), Miracast (registered trademark), or Bluetooth (registeredtrademark).

In the case in which the external equipment OA is wirelessly connectedto the communication section 117, the control section 140 obtains thecontent data from the communication section 117, and then makes theimage display section 20 display the image. In contrast, in the case inwhich the external equipment OA is connected to the interface 114 withwire, the control section 140 obtains the content data from theinterface 114, and then makes the image display section 20 display theimage. The communication section 117 and the interface 114 function as adata acquisition section DA for obtaining the content data from theexternal equipment OA.

The control section 140 is provided with a CPU (not shown) for executingthe programs, a RAM (not shown) for temporality storing the programsexecuted by the CPU and data, and a ROM (not shown) for storing a basiccontrol program executed by the CPU and data in a nonvolatile manner.The control section 140 retrieves and then executes the computerprograms stored in the storage section 120 to thereby function as anoperating system (OS) 150, an image processing section 160, and adisplay control section 170. Further, the control section 140 executesthe programs described above to thereby function as a communicationcontrol section 181, a received information processing section 182, anoperation detection section 183, an operation period display controlsection 184, a transmission image acquisition section 185, a pointinginformation generation section 186, and a sound processing section 187.

The image processing section 160 obtains the image signal included inthe content. The image processing section 160 separates sync signalssuch as a vertical sync signal VSync and a horizontal sync signal HSyncfrom the image signal thus obtained. Further, the image processingsection 160 generates a clock signal PCLK using a phase locked loop(PLL) circuit or the like (not shown) in accordance with the periods ofthe vertical sync signal VSync and the horizontal sync signal HSync thusseparated. The image processing section 160 converts the analog imagesignal, from which the sync signals are separated, into a digital imagesignal using an A/D conversion circuit or the like (not shown). Theimage processing section 160 stores the digital image signal obtained bythe conversion into the RAM in the control section 140 frame by frame asthe image data (Data in the drawing) of the target image. The image datais, for example, RGB data.

It should be noted that it is also possible for the image processingsection 160 to perform a resolution conversion process for convertingthe resolution of the image data into the resolution suitable for theright display drive section 22 and the left display drive section 24 ifnecessary. Further, it is also possible for the image processing section160 to perform an image adjustment process for adjusting the luminanceand the chroma of the image data, a 2D/3D conversion process forgenerating 2D image data from 3D image data or generating 3D image datafrom 2D image data, and so on.

The image processing section 160 transmits each of the clock signalPCLK, the vertical sync signal VSync, the horizontal sync signal HSync,and the image data Data stored in the RAM via each of the transmittingsections 51, 52. The transmitting sections 51, 52 each function as atransceiver to perform serial transmission between the control device10A and the image display section 20. It should be noted that the imagedata Data transmitted via the transmitting section 51 is referred to as“right eye image data,” and the image data Data transmitted via thetransmitting section 52 is referred to as “left eye image data.”

The display control section 170 generates control signals forcontrolling the right display drive section 22 and the left displaydrive section 24, and controls generation and emission of the imagelight by each of the right display drive section 22 and the left displaydrive section 24 using the control signals. Specifically, the displaycontrol section 170 controls ON/OFF of the drive of the right LCD 241 bya right LCD control section 211 and ON/OFF of the drive of the rightbacklight 221 by a right backlight control section 201. Further, thedisplay control section 170 controls ON/OFF of the drive of the left LCD242 by a left LCD control section 212 and ON/OFF of the drive of theleft backlight 222 by a left backlight control section 202.

The communication control section 181 controls the communication section117 to thereby control the communication with the HMD 100B. Thecommunication control section 181 receives work information from the HMD100B, and then outputs the work information thus received to thereceived information processing section 182. Further, the communicationcontrol section 181 performs a process of transmitting the pointinginformation generated by the pointing information generation section 186to the HMD 100B.

The received information processing section 182 analyzes the workinformation received by the communication control section 181 to extractthe image data included in the work information. The receivedinformation processing section 182 detects the image of the work objectfrom the image data included in the work information using, for example,the detecting data 122 stored in the storage section 120.

The operation detection section 183 detects an operation of thecommander wearing the HMD 100A. The operation detection section 183detects the operation of the operation section 135 based on theoperation data input from the input information acquisition section 110.Further, the operation detection section 183 detects the operation ofthe commander wearing the HMD 100A using the camera 61. In this case,the operation detection section 183 detects the image of the pointingobject used by the commander from the taken image data of the camera 61.The pointing objects used by the commander include, for example, an arm,a hand, a finger of the commander, and a pointer rod. The operationdetection section 183 identifies the pointing position of the pointingobject based on the position of the image of the pointing object in thetaken image. Further, it is also possible for the operation detectionsection 183 to detect the operation in the case in which the pointingposition thus identified forms a trajectory corresponding to a patternset in advance. This action is a so-called a gesture manipulation inwhich the commander moves the pointing object to provide differentinstructions using the types of the movement.

The operation period display control section 184 controls the display ofthe image display section 20 in the period in which the operationdetection section 183 detects the operation. The operation perioddisplay control section 184 retrieves the menu image data 123 and so onfrom the storage section 120, then controls the image processing section160 and the display control section 170 to display the menu image and soon. Thus, in the case in which, for example, the commander performs theoperation using the pointing object or the operation section 135, acharacter or an image to be an index or an indication of the motion ofthe pointing object, or a menu bar image for making the position inputpossible can be displayed on the image display section 20.

The transmission image acquisition section 185 obtains the image of thepointing object to be included in the pointing information to betransmitted to the HMD 100B. In the case of taking the image of thepointing object using the camera 61, the transmission image acquisitionsection 185 extracts the image of the pointing object from the takenimage of the camera 61 to thereby obtain the image of the pointingobject. Further, in the case in which it is set by the operation of thecommander or the previous setting that the image of the pointing objectis obtained from the storage section 120, the transmission imageacquisition section 185 obtains the transmission image data 124.

The pointing information generation section 186 generates the pointinginformation to be transmitted to the HMD 100B. The pointing informationis the data to be transmitted by the HMD 100A to the HMD 100B, andincludes either one or both of image data and sound data of the content,image data of the pointing object, sound data of the commander generatedby the sound processing section 187, and data representing the positionor the motion of the pointing object in the present embodiment. Thesound processing section 187 obtains a sound signal included in thecontent, amplifies the sound signal thus obtained, and then outputs theresult to the right earphone 32 and the left earphone 34. Further, thesound processing section 187 obtains the sound collected by themicrophone 63, and then converts the sound into digital sound data. Itis also possible for the sound processing section 187 to perform aprocess set in advance on the digital sound data.

The image display section 20 is provided with the camera 61 as describedabove and a distance sensor 64. Further, the image display section 20 isprovided with an interface 25, the right display drive section 22, theleft display drive section 24, the right light guide plate 261 as theright optical image display section 26, the left light guide plate 262as the left optical image display section 28, a nine-axis sensor 66, andan eye sensor 68.

The nine-axis sensor 66 is a motion sensor for detecting acceleration (3axes), angular velocities (3 axes), and geomagnetisms (3 axes). In thecase in which the image display section 20 is mounted on the head of theuser, it is possible for the control section 140 to detect the motion ofthe head of the user based on the detection values of the nine-axissensor 66. For example, it is possible for the control section 140 toestimate the level of the tilt and the direction of the tilt of theimage display section 20 based on the detection values of the nine-axissensor 66.

The interface 25 is provided with a connector to which the right cord 42and the left cord 44 are connected. The interface 25 outputs the clocksignal PCLK, the vertical sync signal VSync, the horizontal sync signalHSync, and the image data Data transmitted from the transmitting section51 to the corresponding receiving sections (Rx) 53, 54. Further, theinterface 25 outputs the control signals, which are transmitted from thedisplay control section 170, to the receiving sections 53, 54, the rightbacklight control section 201 or the left backlight control section 202corresponding to the control signals.

Further, the interface 25 is an interface for connecting the camera 61,the distance sensor 64, the nine-axis sensor 66, and the eye sensor 68to each other. The taken image data of the camera 61, the detectionresult of the distance sensor 64, the detection result of theacceleration (three axes), the angular velocity (three axes), andgeomagnetism (three axes) provided by the nine-axis sensor 66, and thedetection result of the eye sensor 68 are transmitted to the controlsection 140 via the interface 25.

The right display drive section 22 is provided with the right backlight221, the right LCD 241, and the right projection optical system 251described above. Further, the right display drive section 22 is providedwith the receiving section 53, the right backlight (BL) control section201 for controlling the right backlight (BL) 221, and the right LCDcontrol section 211 for driving the right LCD 241.

The receiving section 53 acts as a receiver corresponding to thetransmitting section 51, and performs the serial transmission betweenthe control device 10A and the image display section 20. The rightbacklight control section 201 drives the right backlight 221 based onthe control signal input to the right backlight control section 201. Theright LCD control section 211 drives the right LCD 241 based on theclock signal PCLK, the vertical sync signal VSync, the horizontal syncsignal HSync, and the right-eye image data Data input via the receivingsection 53.

The left display drive section 24 has substantially the sameconfiguration as that of the right display drive section 22. The leftdisplay drive section 24 is provided with the left backlight 222, theleft LCD 242, and the left projection optical system 252 describedabove. Further, the left display drive section 24 is provided with thereceiving section 54, the left backlight control section 202 for drivingthe left backlight 222, and the left LCD control section 212 for drivingthe left LCD 242.

The receiving section 54 acts as a receiver corresponding to thetransmitting section 52, and performs the serial transmission betweenthe control device 10A and the image display section 20. The leftbacklight control section 202 drives the left backlight 222 based on thecontrol signal input to the left backlight control section 202. The leftLCD control section 212 drives the left LCD 242 based on the clocksignal PCLK, the vertical sync signal VSync, the horizontal sync signalHSync, and the left-eye image data Data input via the receiving section54.

It should be noted that the right backlight control section 201, theright LCD control section 211, the right backlight 221, and the rightLCD 241 are also collectively referred to as a right “image lightgeneration section.” Similarly, the left backlight control section 202,the left LCD control section 212, the left backlight 222, and the leftLCD 242 are also collectively referred to as a left “image lightgeneration section.”

As shown in FIG. 5, the HMD 100B has a configuration having the imagedisplay section 20, which is explained with reference to FIG. 4, and thecontrol device 10B connected to each other. The control device 10B has acontrol section 300 instead of the control section 140 (FIG. 4), and hasa storage section 340 instead of the storage section 120 (FIG. 4). Otherconstituents are common to the control device 10B and the control device10A as shown in FIG. 4.

The storage section 340 is a nonvolatile storage device similar to thestorage section 120, and stores a variety of computer programs, and thedata related to these programs. Further, it is also possible for thestorage section 340 to store the data of a still image and a movingimage to be displayed on the image display section 20.

The storage section 340 stores configuration data 341. The configurationdata 341 includes a variety of types of setting values used by thecontrol section 300. The setting values included in the configurationdata 341 can be values input by the operation of the operation section135 in advance, or it is also possible to receive the setting valuesfrom the external equipment OA or other devices (not shown) via thecommunication section 117 or the interface 114, and then store thesetting values.

Further, the storage section 340 is a nonvolatile storage device, andstores a variety of computer programs to be executed by the controlsection 300, and the data related to these programs. Further, thestorage section 340 stores detecting data 343, menu image data 345,transmission image data 124, and instruction content data 125.

The detecting data 343 is the data used for detecting an image of a workobject from the image data. The detecting data 343 includes, forexample, the data used for the process of extracting the image of thework object from the image data, and more specifically includes the dataof the feature amount of the image of the work object. The featureamount can also be the feature amount of the color or the feature amountof the shape, and it is also possible to include a plurality of featureamounts in accordance with the number of colors and the resolution ofthe image data as the processing target. The detecting data 343 storedin the storage section 340 can be the same as the configuration data121, or can also be data different from the configuration data 121.

The menu image data 345 is the data of the menu image displayed by theimage processing section 303 and the display control section 305.

Similarly to the control section 140, the control section 300 isprovided with a CPU (not shown) for executing the programs, a RAM (notshown) for temporality storing the programs executed by the CPU anddata, and a ROM (not shown) for storing a basic control program executedby the CPU and data in a nonvolatile manner. The control section 300retrieves and then executes the computer programs stored in the storagesection 340 to thereby function as an operating system (OS) 301, animage processing section 303, and a display control section 305.Further, the control section 300 functions as a communication controlsection 311, a taken image control section 313, a work informationgeneration section 315, a received information analysis section 317, apointing detection section 319, a display position calculation section321, an operation detection section 323, a display informationgeneration section 325, and a sound processing section 329.

The image processing section 303 obtains the image signal included inthe content. The image processing section 303 separates the sync signalssuch as the vertical sync signal VSync and the horizontal sync signalHSync from the image signal thus obtained. Further, the image processingsection 303 generates the clock signal PCLK using a phase locked loop(PLL) circuit or the like (not shown) in accordance with the periods ofthe vertical sync signal VSync and the horizontal sync signal HSync thusseparated. The image processing section 303 converts the analog imagesignal, from which the sync signals are separated, into a digital imagesignal using an A/D conversion circuit or the like (not shown). Theimage processing section 303 stores the digital image signal obtained bythe conversion into the RAM in the control section 300 frame by frame asthe image data (Data in the drawing) of the target image. The image datais, for example, RGB data. It should be noted that it is also possiblefor the image processing section 303 to perform a resolution conversionprocess for converting the resolution of the image data into theresolution suitable for the right display drive section 22 and the leftdisplay drive section 24 if necessary. Further, it is also possible forthe image processing section 303 to perform an image adjustment processfor adjusting the luminance and the chroma of the image data, a 2D/3Dconversion process for generating 2D image data from 3D image data orgenerating 3D image data from 2D image data, and so on. The imageprocessing section 303 transmits each of the clock signal PCLK, thevertical sync signal VSync, the horizontal sync signal HSync, and theimage data Data stored in the RAM via each of the transmitting sections51, 52. The transmitting sections 51, 52 each function as a transceiverto perform serial transmission between the control device 10B and theimage display section 20. It should be noted that the image data Datatransmitted via the transmitting section 51 is referred to as “right eyeimage data,” and the image data Data transmitted via the transmittingsection 52 is referred to as “left eye image data.”

The display control section 305 generates control signals forcontrolling the right display drive section 22 and the left displaydrive section 24, and controls generation and emission of the imagelight by each of the right display drive section 22 and the left displaydrive section 24 using the control signals. Specifically, the displaycontrol section 305 controls ON/OFF of the drive of the right LCD 241 bythe right LCD control section 211 and ON/OFF of the drive of the rightbacklight 221 by the right backlight control section 201. Further, thedisplay control section 305 controls ON/OFF of the drive of the left LCD242 by the left LCD control section 212 and ON/OFF of the drive of theleft backlight 222 by the left backlight control section 202.

The communication control section 311 controls the communication section117 to thereby control the communication with the HMD 100A. Thecommunication control section 311 performs a process of transmitting thedata, which includes the transmission information generated by the workinformation generation section 315, to the HMD 100A. Further, thecommunication control section 311 receives the data transmitted by theHMD 100A, and then outputs the data thus received to the receivedinformation analysis section 317. The taken image control section 313controls the camera 61 to perform imaging to thereby obtain the takenimage data. The work information generation section 315 obtains the datato be transmitted to the HMD 100A, and then generates the workinformation based on the data thus obtained. The work informationgeneration section 315 obtains the taken image data taken by the camera61. Further, it is also possible for the work information generationsection 315 to obtain the detection value of at least a part of thesensors consisting of the distance sensor 64, the nine-axis sensor 66,and the eye sensor 68, and in this case, there is generated the workinformation including the taken image data taken by the camera 61 andthe detection value of the sensor. Although the sensors, the detectionvalues of which are obtained by the work information generation section315, are set in advance, it is also possible for the HMD 100A totransmit a setting command to the HMD 100B to designate the detectionvalues to be obtained. Further, it is also possible for the workinformation generation section 315 to generate not only the taken imagedata taken by the camera 61 itself, but also the work informationincluding the image data generated from the taken image data.

The received information analysis section 317 analyzes the data receivedby the communication control section 311. In the present embodiment, thereceived information analysis section 317 extracts the image data of thepointing object, the image data, the sound data, and so on of thecontent included in the pointing information received by thecommunication control section 311. Further, the received informationanalysis section 317 extracts the data representing the position and themotion of the pointing object from the pointing information received bythe communication control section 311. The pointing detection section319 detects the data representing the fact that the operation using thepointing object is performed out of the data extracted by the receivedinformation analysis section 317. It is possible for the commander tooperate the HMD 100A to perform the position pointing operation usingthe pointing object, and to control the HMD 100B using the positionpointing operation. In this case, the data related to the positionpointing operation included in the pointing information transmitted bythe HMD 100A is detected by the HMD 100B as an operation of a GUI. Thedata for designating whether or not the operation of the commander isdetected as the operation of the GUI is included in the pointinginformation transmitted by the HMD 100A, and is detected by the pointingdetection section 319.

The display position calculation section 321 calculates the position, atwhich the image is displayed, based on the image data extracted by thereceived information analysis section 317. The image data extracted bythe received information analysis section 317 corresponds to the imagedata of the pointing object, the image data of the content, and otherimage data. The display position calculation section 321 calculates thedisplay position of each of the display position of the image of thepointing object, the display position of the image data of the content,and the display positions of other images based on the data describedabove. Further, it is also possible for the display position calculationsection 321 to calculate a display size of each of the images. Further,in the case in which the pointing detection section 319 detects the dataof instructing to use the operation of the commander as the operation ofthe GUI, the display position calculation section 321 calculates thedisplay position of the menu image used for performing the GUIoperation.

In the case in which the data representing the position and the motionof the pointing object extracted by the received information analysissection 317 is detected as the operation of the GUI, the operationdetection section 323 detects the operation. The operation detectionsection 323 detects the operation of the GUI based on the displaypositions of the icon of the menu image and so on calculated by thedisplay position calculation section 321 and the display position of theimage of the pointing object.

The display information generation section 325 generates the displayinformation of the screen displayed by the image display section 20. Thedisplay information generation section 325 disposes the images based onthe image data extracted by the received information analysis section317 at the display positions calculated by the display positioncalculation section 321 to generate the display informationcorresponding to the entire screen. The display information generated bythe display information generation section 325 is transmitted to theimage display section 20 due to the actions of the display controlsection 170 and the display control section 305, and is displayed by theimage display section 20.

The sound processing section 329 obtains a sound signal included in thecontent, amplifies the sound signal thus obtained, and then outputs theresult to the right earphone 32 and the left earphone 34. Further, thesound processing section 329 obtains the sound collected by themicrophone 63, and then converts the sound into digital sound data. Itis also possible for the sound processing section 329 to perform aprocess set in advance on the digital sound data.

In the display system 1 having the configuration described above, theHMD 100B corresponds to a display device and a first display device, andthe HMD 100A corresponds to a second display device. The image displaysection 20 provided to the HMD 100B corresponds to a first displaysection, the communication section 117 corresponds to a firstcommunication section, the control section 300 corresponds to a controlsection and a first control section, and the camera 61 corresponds to animaging section. Further, the image display section 20 provided to theHMD 100A corresponds to a second display section, the communicationsection 117 corresponds to a second communication section, and thecontrol section 140 corresponds to a second control section.

FIGS. 6A and 6B are flowcharts showing an action of the display system1, wherein FIG. 6A shows an action of the HMD 100A, and FIG. 6B shows anaction of the HMD 100B. FIGS. 6A and 6B show an example in which one HMD100A and one HMD 100B act together. In the HMD 100B, the imaging controlsection 313 controls the camera 61 to perform imaging to thereby obtain(step S21) the taken image data, and the work information generationsection 315 obtains (step S22) the detection values of the respectivesensors.

Subsequently, the work information generation section 315 generates thework information including the taken image data in the step S21, and thedata of the detection values obtained in the step S22, and then thecommunication control section 311 transmits the data to the HMD 100A(step S23).

In the HMD 100A, the communication control section 181 receives the workinformation having been transmitted from the HMD 100B, and then thereceived information processing section 182 analyzes the workinformation thus received (step S11). The received informationprocessing section 182 extracts the data of the taken image from thework information thus received to make the image processing section 160and the display control section 170 display the taken image (step S12).Further, in the case in which the detection value of the sensor isincluded in the work information received by the communication controlsection 181, it is also possible for the received information processingsection 182 to generate data for displaying the detection value of thesensor so as to be superimposed on the taken image, and then make theimage processing section 160 and the display control section 170 displaythe detection value.

Subsequently, in the HMD 100A, the content is selected (step S13) due tothe operation of the commander, and the operation for the commander togive an instruction and so onto the worker is performed (step S14). Inthe step S13, a supporting content including a text and an image, forexample, is selected. In the step S14, an operation of pointing theposition to which the worker is made to pay attention is performed withrespect to, for example, the image, which has been received by thecommunication control section 181, and is being displayed. The operationin the step S14 is an operation on the operation section 135, or theoperation of the commander moving the pointing object in front of theHMD 100A. This operation is detected (step S15) by the operationdetection section 183, and the operation detection section 183 generatesthe data of the operation thus detected.

The operation detection section 183 generates (step S16) the data formaking the operation position and the operation trajectory of theoperation thus detected correspond to the taken image of the HMD 100Bpresently displayed. Thus, there can be obtained the data representing aspecific position in the view field of the worker. The transmissionimage acquisition section 185 clips the image of the pointing objectfrom the taken image taken by the camera 61 during the operation in thestep S14, or retrieves the transmission image data 124 from the storagesection 120, to obtain the transmission image (step S17).

The pointing information generation section 186 generates and thentransmits the pointing information including the image data of the imageof the pointing object obtained by the transmission image acquisitionsection 185, the data representing the operation position and thetrajectory of the operation, and the data for making the operationposition and the trajectory of the operation correspond to the takenimage of the HMD 100B (step S18).

The control section 140 determines (step S19) whether or not atermination condition of the action has been fulfilled, and if thetermination condition has not been fulfilled (NO in the step S19), theprocess returns to the step S11. Further, if the termination conditionhas been fulfilled (Yes in the step S19), the present process isterminated. As the termination condition, there can be cited, forexample, the fact that the termination of the action or shutdown of theHMD 100A is instructed by the operation on the operation section 135.

The HMD 100B receives (step S24) the pointing information transmitted bythe HMD 100A with the communication control section 311, and then thereceived information analysis section 317 analyzes the pointinginformation thus received to extract (step S25) the image data.Subsequently, the display position calculation section 321 calculatesthe positions at which the images are displayed based on the respectiveimage data (step S26). Further, the display information generationsection 325 generates the display information with the images disposedat the display positions thus calculated, and then makes the imageprocessing section 303 and the display control section 305 display theimages (step S27).

The control section 300 determines (step S28) whether or not atermination condition of the action has been fulfilled, and if thetermination condition has not been fulfilled (NO in the step S28), theprocess returns to the step S23. Further, if the termination conditionhas been fulfilled (Yes in the step S28), the present process isterminated. As the termination condition, there can be cited, forexample, the fact that the termination of the action or shutdown of theHMD 100B is instructed by the operation on the operation section 135.FIGS. 7A through 7D are diagrams each showing a display example in theHMD 100A, 100B, wherein FIG. 7A shows the view field VRA of thecommander wearing the HMD 100A, and FIG. 7B shows the view field VRB1 ofthe worker wearing the HMD 100B. Further, FIG. 7C shows another exampleof the view field VRA of the commander wearing the HMD 100A, and FIG. 7Dshows the view field VRB2 of another worker wearing the HMD 100B. Itshould be noted that FIGS. 7A through 7D each show an example of theview field of the right eye of the commander or the worker, and the viewfield of the left eye is the same as or bilaterally symmetrical with therespective drawings, and is therefore omitted from the graphicaldescription.

Here, in the pointing information (guide information) generated by thepointing information generation section 186, the information forassociating the operation and the work information with each other, orthe specific configuration of the information to be associated with thetaken image included in the work information is not particularlylimited. Although in the example described above, the explanation ispresented assuming that the operation position and the trajectory of theoperation are the data to be associated with the taken image of the HMD100B, it is also possible to simply use the operation position and thetrajectory of the operation as the information described above. Further,it is also possible to adopt the data for making the operation positionand the trajectory of the operation correspond to the taken image of theHMD 100B using the correspondence of the file manes, types of the data,ID, attribute, or the positions, and it is sufficient for the data to bethe information making it possible to recognize the correspondence as aresult.

In the view field VRA of the commander shown in FIG. 7A, the areacorresponding to the half mirror 261A becomes the display area 401, andthe image to be drawn in the right LCD 241 (FIG. 4) is displayed in thedisplay area 401. In the example shown in FIG. 7A, a normal operationscreen 402 is displayed in the display area 401. The normal operationscreen 402 is a screen displayed using the function of the OS 150 andthe function of an application program running on the OS 150, and is thescreen related to the operation of files and folders in the exampleshown in FIG. 7A.

In the case in which the HMD 100A has received the work information fromthe HMD 100B, the HMD 100A displays the taken image of the HMD 100B asshown in FIG. 7A (the step S12 shown in FIG. 6A). The display area 403for displaying the taken image is disposed in the display area 401, thetaken image is displayed in the display area 403. Since the half mirror261A transmits the outside view, the commander visually recognizes thepointing object 2 operated by the commander through the display area401. It should be noted that in the example shown in FIG. 7A, thepointing object 2 is a hand of the commander. The commander performs theoperation of, for example, pointing with the pointing object 2 to theimage 404 of the work object displayed in the display area 401. The HMD100A detects a pointing position 405 pointed by the pointing object 2using the operation detection section 183, and then calculates therelative position between the pointing position 405 and the taken imagepresently displayed in the display area 403. In the view field VRB1 ofthe worker shown in FIG. 7B, the area corresponding to the half mirror261A becomes the display area 451, and the image to be drawn in theright LCD 241 (FIG. 5) is displayed in the display area 451. Since thehalf mirror 261A transmits the outside view, the work object 452 locatedin the view field of the worker is visually recognized through thedisplay area 451. Further, in the display area 451, there is displayedan image 453 of the pointing object pointing the pointing position 454pointed by the commander with the pointing object 2. The image 453 ofthe pointing object is displayed based on the image data include in thepointing information received (the step S24 shown in FIG. 6B) by the HMD100B from the HMD 100A. Further, the position of the pointing position454 is designated by the data included in the pointing information.

As shown in FIGS. 7A and 7B, since the commander wearing the HMD 100Acan see the taken image obtained by taking the image of the view fieldof the worker, it is possible to perform an instruction or the likerelated to the work while observing the view field of the worker.Further, since the pointing position pointed by the commander with thepointing object 2 is detected by the HMD 100A, and the data representingthe pointing position with respect to the taken image is transmitted tothe HMD 100B, it is possible for the worker to see the image 453 of thepointing object so as to be superimposed on the outside view in theactual view field. Therefore, it is possible for the worker to have aninstruction with the image superimposed on the work object actuallyviewed as the outside view. Further, since the image 453 of the pointingobject is displayed based on the data transmitted by the HMD 100A to theHMD 100B, it is possible for the HMD 100A to select which one of thetaken image of the actual pointing object 2 and the image prepared inadvance is used as the image of the pointing object. Here, it is alsopossible to store the image data for displaying the image of thepointing object in the storage section 340 of the HMD 100B in advance,and include the data for designating the image stored in the HMD 100B inthe pointing information transmitted by the HMD 100A to the HMD 100B.

As shown in FIG. 1, the display system 1 can be used with a plurality ofHMD 100B connected to the communication network 4. In this case, the HMD100A is capable of receiving the work information from the plurality ofHMD 100B to display the work information thus received.

FIG. 7C shows an example in which the HMD 100A displays the taken imagesbased on the work information received from the two HMD 100B. In theexample shown in FIG. 7C, two display areas 409, 410 for displaying thetaken images are disposed in the display area 401. In the display area409, there is displayed an image based on the work informationtransmitted by one of the HMD 100B. Further, in the display area 410,there is displayed an image based on the work information transmitted bythe other of the HMD 100B. In the case in which pointing with thepointing object 2 is performed on the images displayed in the displayareas 409, 410, the HMD 100A generates and then transmits the pointinginformation to the HMD 100B having transmitted the work informationcorresponding to the position thus pointed.

Further, in the case in which the display system 1 includes a pluralityof HMD 100B, it is also possible to transmit the pointing information,which has been transmitted by the HMD 100A to one of the HMD 100B, toanother of the HMD 100B. FIG. 7D shows the display example in this case.FIG. 7D shows the display example in the case in which the pointinginformation shown in FIG. 7B as an example is received by another of theHMD 100B.

In the view field VRB2 of another worker wearing the HMD 100B, the image462 of the work object is displayed in the display area 461corresponding to the half mirror 261A, and the image 463 of the pointingobject pointing the pointing position 464 is displayed. In the viewfield VRB2, the work object does not show, but an unrelated outside viewshows unlike the view field VRB1 of the worker shown in FIG. 7B. If thepointing information transmitted by the HMD 100A includes the takenimage of the HMD 100B, it is possible to display the image 462 of thework object in the display area 461 as shown in FIG. 7D. In thisexample, it is possible for other workers to know the content of thepointing information transmitted to either one of the workers.

As explained hereinabove, the display system 1 according to the firstembodiment to which the invention is applied is provided with the HMD100B and the HMD 100A of the head mounted type. The HMD 100B is providedwith the image display section 20 for displaying an image so that theoutside view can visually be recognized, and the camera 61 for taking animage of a range including at least a part of the outside view which canvisually be recognized in the image display section 20. Further, the HMD100B is further provided with the communication section 117 fortransmitting the work information generated based on the taken imagetaken by the camera 61 to the HMD 100A, and receiving the pointinginformation from the HMD 100A, and the control section 300 for makingthe image display section 20 display the image based on the pointinginformation received by the communication section 117. The HMD 100A isprovided with the image display section 20, and the communicationsection 117 for receiving the work information from the HMD 100B andtransmitting the pointing information to the HMD 100B.

Further, the HMD 100A is provided with the display control section 305for making the image display section 20 display an image based on thework information received by the communication section 117, theoperation detection section 183, and the control section 140 forgenerating and then transmitting the pointing information including theinformation of associating the operation detected by the operationdetection section 183 and the work information with each other.

Therefore, the HMD 100B transmits the work information based on thetaken image thus taken to the HMD 100A, the HMD 100A displays the imagebased on the work information, and the pointing information includingthe information for associating the operation during the period ofdisplaying the image and the work information with each other istransmitted to the HMD 100B. Therefore, since the information related tothe operation associated with the taken image in the HMD 100B can beprovided to the HMD 100B using the operation of the HMD 100A, it ispossible to transmit a large amount of information to the worker wearingthe HMD 100B in an easy-to-understand manner.

Further, the control section 300 of the HMD 100B generates the workinformation including the taken image taken by the camera 61, and theoperation detection section 183 provided to the HMD 100A detects theposition pointing operation. The control section 140 generates thepointing information including the information for associating thepointing position of the position pointing operation detected by theoperation detection section 183 with the taken image included in thework information. Therefore, it is possible to make the pointingposition of the position pointing operation performed in the HMD 100Acorrespond to the taken image of the HMD 100B. Therefore, it is possibleto transmit the content of the position pointing operation performed inthe HMD 100A to the worker wearing the HMD 100B.

Further, the control section 140 generates pointing informationincluding the information related to the pointing position of theposition pointing operation detected by the operation detection section183 and the image of the pointing object. The control section 300receives the pointing information using the communication section 117,and makes the image of the pointing object be displayed at the positioncorresponding to the pointing position included in the pointinginformation thus received. Since the HMD 100B displays the image of thepointing object in accordance with the pointing position due to theposition pointing operation in the HMD 100A, it is possible to point theposition to the worker wearing the HMD 100B using the operation in theHMD 100A. An application example of the display system 1 will beexplained citing a specific example.

The commander can use the display system 1 for the purpose of providingthe information related to the operation of the equipment in such amanner that the operation of the equipment is explained and guided fromthe remote location to support the work in the case in which, forexample, the worker performs a work of operating the equipment.Specifically, the display system 1 can be applied in the case of settinga personal computer as the work object, and performing setup or anoperation of an application program. Further, the display system 1 canbe applied in the case of, for example, setting a machine such as aworking machine, an industrial machine, or a variety of types ofconsumer-electronics devices as the work object, and performing anoperation or maintenance of the machine. Specifically, in the case inwhich the worker for performing the maintenance of a variety of types ofmachines visits the installation site of the machine as the work objectat the request of the user, and performs the maintenance, the commandercan use the display system 1 for the purpose of supporting the workerlow in proficiency level.

In such a case, the worker located in the working site B in the remotelocation and the commander located in the command site A such as aservice center have the information in common, and it is possible forthe commander in the command site A to make the HMD 100B display the ARimage to change pointing and display. Further, the support can beachieved by showing the operation with a hand or a finger of thecommander located in the command site A as a virtual image using the HMD100B. In this case, it is effective to place the same equipment as thework object in the working site B in front of the commander in thecommand site A, and make an explanation while actually operating theequipment. It is possible for the commander to make an instruction andteaching while performing the position pointing operation to thespecific region of the actual object in the actual place, and to makethe efficient support with a simple operation.

As another application example, by including image data and a text of aname card in the pointing information transmitted from the HMD 100A, itis possible to show the name card to the worker. In this case byincluding the image data of an image resembling an arm of the commanderin the pointing information, it is possible to display the arm holdingout the name card on the HMD 100B to realize virtual name card exchange.

It is also possible to perform the name card exchange between thecommander in the command site A and the worker in the working site B,and then perform the action explained in the above description of theembodiment.

Further, by the HMD 100A including the data of the content in thepointing information and then transmitting the pointing information tothe HMD 100B, it is possible to display the content including a text andan image on the image display section 20 of the HMD 100B. In this case,it is preferable that the display position of the text and the image ofthe content is set to a position not hindering the visual recognition ofthe work object in accordance with the actual position of the workobject visually recognized through the image display section 20.

Further, it is also possible for the sound data as the data of thecontent to be included in the pointing information, and to betransmitted from the HMD 100A to the HMD 100B. The sound data of thecontent can be the data stored in the storage section 120 in advance, orcan also be the sound data obtained by collecting the voice of thecommander with the microphone 63. Further, by using an avatar acting asthe commander, or an image of the actual face, upper body, or whole bodyof the commander as the image of the pointing object, it is alsopossible to realize servicing intended for the consumer instead of theworker. In this case, the commander is changed to a staff of a callcenter, a private tutor, a tutoring staff of a cram school, or the like.As the purpose of this system, there can be cited a guide or training ofan installation or an operation of software of electronic equipment,language teaching, coaching for an examination such as a certificationexamination, a course in a public qualification such as a driverlicense, coaching for sports, and so on. Further, for example, in acourse of a foreign language conversation, it is possible that thestudent wears the HMD 100B, the teacher wears the HMD 100A, the HMD 100Bperforms the AR display so that the shape of the mouth of the teacher issuperimposed on the image of the mouth of the student reflected on amirror, and thus, the pronunciation can visually be taught.

Further, in the case of coaching for study or a sport, the HMD 100A isused in the command site A on the coaching side, and the HMD 100B isused in the working site B on the coached side. Including such a case asdescribed above, the command site A can be called a “management site”where a person performing coaching or management is located, and the HMD100A can be called a management device. Further, the working sites B, Ccan be called an “execution site” where a person subject to coaching orteaching and executing leaning, or a person executing the work islocated, and the HMD 100B, 100C can be called an execution-side device.Further, the work information transmitted from each of the HMD 100B,100C to the HMD 100A can be called guide target information orexecution-side information, and the pointing information can be calledguide information, coaching information or management information.

As a specific example of coaching, teaching and coaching are performedusing the content displayed on the HMD 100B or the HMD 100C worn by theperson subject to coaching with respect to the learning subjects such asmathematics, English, Japanese, social science, character stroke order,and so on. The same is applied to the case of using the system for thepurpose of guiding the operation method of equipment such as a personalcomputer. Further, in the case of coaching English conversation, byshooting the mouth of the teacher wearing the HMD 100A and the mouth ofthe student (learner, performer) wearing the HMD 100B, 100C, and thentransmitting and receiving the work information including the shotimages and the pointing information, more effective learning andcoaching can be performed.

As another application example, by the commander wearing the HMD 100Aperforming the guide of the way, peripheral building, and so on to aperson wearing the HMD 100B, the system can be applied to, for example,a tour guide or a guide post in a tourist spot.

Further, in the HMD 100A, 100B, and 100C, it is possible to displayspecific information in a conspicuous manner such as pop-up, blinking,or display in a fluorescent color. Further, it is also possible tochange the display color or the display luminance of the image to bedisplayed so that the visibility of the outside view transmitting theimage display section 20 is enhanced. On this occasion, since thevisibility of the image displayed by the image display section 20 dropsrelatively to the outside view, there is an effect of making thecharacters and the images included in the display image seem ancillary.

In the display system 1, the taken image at the viewpoint of the workerwearing the HMD 100B can be seen in the HMD 100A in the remote location,and the worker can see the content of the position pointing operationperformed by the commander in the HMD 100A. The AR display of thecontent of the position pointing operation can be achieved using, forexample, the image 453 of the pointing object. Specifically, in the HMD100B, the image 453 of the pointing object is displayed so as to overlapthe work object 452 viewed by the worker as the outside view, and due tothe positions of the image 453 of the pointing object and the workobject 452, the display effect as augmented reality (AR) can beexpected. In order to enhance the effect of the AR display, it ispossible for the HMD 100B to display the images having the parallaxbetween the right display drive section 22 and the left display drivesection 24 when displaying the image 453 of the pointing object and soon. In this case, the control section 300 determines the direction andthe level of the parallax in accordance with the type of the work objectand the work content. The display information generation section 325 ofthe control section 300 generates the images having the parallax basedon the pointing information, and then makes the image processing section303 and the display control section 305 display the images. Further, itis also possible for the control section 300 to detect the distance tothe work object using the distance sensor 64. In this case, it ispossible for the control section 300 to calculate the direction and thelevel of the parallax of the images of the pointing object on which theAR display is performed based on the distance to the work object thusdetected. For example, it is also possible to provide the parallax basedon the distance condition set in advance. In this case, the imagesshowing the range of the reach of the worker wearing the HMD 100B, 100Vare visually recognized in a stereoscopic manner. Further, it is alsopossible to arrange that the images provided with the parallax aredisplayed in the HMD 100A so that the commander can visually recognizethe images in a stereoscopic manner.

Further, as explained with reference to FIGS. 7C and 7D, it is possiblefor the worker wearing the HMD 100B in the working site C to see theworking operation of the worker in the working site B from the remotelocation as the AR image. Thus, it is possible to have the content ofthe work support such as the work procedure in common.

Further, if the display of the HMD 100A and the taken image of the HMD100B, or the taken images of two or more HMD 100B are displayed in thetwo or more display areas using the HMD 100A, the commander can visuallyrecognize a lot of circumstances.

Here, as shown in FIG. 7C, in the case of displaying the imagescorresponding to two or more HMD 100B in two or more display areas, itis also possible to adopt a configuration in which the shapes and thecolors of the pointers or the like in the respective display areas aremade different from each other to thereby prevent the confusion of thedisplay areas.

Besides the above, the application range of the display system 1 is notat all limited, and applications to other purposes can obviously bemade.

Second Embodiment

FIGS. 8A, 8B, and 9 are flowcharts showing an action of the displaysystem 1 according to a second embodiment. FIG. 8A shows an action ofthe HMD 100A, and FIG. 8B shows an action of the HMD 100B. Further,FIGS. 10A through 10D are diagrams each showing a display example of theHMD 100A, 100B according to the second embodiment.

Since the display system 1 according to the second embodiment isconfigured similarly to the display system 1 explained in the abovedescription of the first embodiment, graphical description and theexplanation of the constituents of the display system 1 will be omitted.Further, in the flowcharts shown in FIGS. 8A, 8B, and 9, the actionscommon to the first embodiment and the second embodiment and alreadyexplained in the description of the first embodiment (FIGS. 6A and 6B)are denoted with the same step numbers.

In the second embodiment, the point that the work information includingthe taken image taken by the camera 61 of the HMD 100B is transmitted tothe HMD 100A, and is displayed in the HMD 100A is substantially the sameas in the first embodiment. In the second embodiment, the HMD 100B canbe controlled by the position pointing operation performed in the HMD100A. In the actions of the HMD 100A shown in FIG. 8A, the steps S11through S17 are as explained in the above description of the firstembodiment.

In the step S17, after the transmission image acquisition section 185obtains the transmission image, whether or not the menu operation to theHMD 100B is performed is designated by the operation of the operationsection 135 (step S31). Then, the pointing information generationsection 186 generates the pointing information including the data fordesignating whether or not the menu operation is performed, and thentransmits the pointing information to the HMD 100B (step S32). Thepointing information generated in the step S32 includes the image dataof the image of the pointing object obtained by the transmission imageacquisition section 185, the data representing the operation positionand the trajectory of the operation, and the data for making theoperation position and the trajectory of the operation correspond to thetaken image of the HMD 100B.

In the actions of the HMD 100B shown in FIG. 8B, the steps S21 throughS27 are as explained in the above description of the first embodiment.

The HMD 100B receives the pointing information transmitted by the HMD100A with the communication control section 311, and then generates thedisplay information based on the pointing information to make the imageprocessing section 303 and the display control section 305 display thedisplay information (step S27).

Subsequently, the control section 300 executes the operation detectionprocess (step S35).

FIG. 9 is a flowchart showing the operation detection process executedby the HMD 100B in detail.

In the operation detection process, the operation detection section 323of the HMD 100B extracts the data designating whether or not the menuoperation is performed from the pointing information received in thestep S24, and then determines whether or not the menu operation isperformed based on the data (step S41). In the case in which it has beendetermined that the menu operation is not performed (NO in the stepS41), the process proceeds to the step S28 shown in FIG. 8B.

In the case in which it has been determined that the menu operation isperformed (YES in the step S41), the operation detection section 323determines the display position of the menu image, and then displays themenu image (step S42). Here, the display position of the menu image canbe designated with the data included in the pointing informationtransmitted by the HMD 100A, or can also be a position set in advance.Further, it is also possible for the operation detection section 323 toautomatically determine the position. Subsequently, the operationdetection section 323 obtains (step S43) the data representing theoperation position from the pointing information, and then determines(step S44) whether or not the operation position thus obtained is theposition corresponding to the menu image.

Here, in the case in which it is determined that the operation positionis not the position corresponding to the menu image (NO in the stepS44), the control section 300 makes the transition to the step S28.

In the case in which the operation position is the positioncorresponding to the menu image (YES in the step S44), the operationdetection section 323 detects the operation to the menu image (stepS45). In the step S45, the operation detection section 323 identifiesthe operating image corresponding to the operation position out of theoperating images such as a plurality of icons disposed in the menuimage. The control section 300 executes (step S46) the action associatedwith the operating image identified by the operation detection section323, and then makes the transition to the step S28 after completing theaction.

FIG. 10A shows the view field VRA of the commander wearing the HMD 100A,and FIG. 10B shows the view field VRB1 of the worker wearing the HMD100B. Further FIG. 10C shows another example of the view field VRA ofthe commander wearing the HMD 100A, and FIG. 10D shows the view fieldVRB2 of another worker wearing the HMD 100B. It should be noted thatFIGS. 10A through 10D each show an example of the view field of theright eye of the commander or the worker, and the view field of the lefteye is the same as or bilaterally symmetrical with the respectivedrawings, and is therefore omitted from the graphical description.

In the view field VRA of the commander shown in FIG. 10A, the normaloperation screen 402 is displayed in the display area 401, and the takenimage of the HMD 100B is displayed in the display area 403 fordisplaying the taken image. In the case in which it is designated toperform the menu operation using the operation of the operation section135, the menu image 411 is displayed in the display area 401. The menuimage 411 is not the image for performing the operation to the HMD 100A,but is the image displayed for performing the operation to the HMD 100B.In the menu image 411, there is disposed, for example, a plurality oficons (the operating images) each having a button shape. The icons areassociated with the actions performed in the HMD 100B, respectively. Asthe actions of this kind, there can be cited, for example, displayexpansion/contraction, drawing start/stop, designation of a figure to bedrawn, and designation of a drawing color. The menu image 411 isdisplayed by the operation period display control section 184 based onthe menu image data 123.

When the operation using the pointing object 2 is performed on the menuimage 411, the operation detection section 183 detects the operation toidentify the operation position. Further, the operation detectionsection 183 obtains the positions of the menu image 411 during theoperation and the operation sections such as the icons included in themenu image 411. In the case in which the operation detection section 183detects the operation in the state in which the pointing informationgeneration section 186 displays the menu image 411, the pointinginformation generation section 186 includes the data pointing the menuoperation in the pointing information. Further, the pointing informationgeneration section 186 generates pointing information including theoperation position of the operation detected by the operation detectionsection 183, and the positions of the menu image 411 and so on obtainedby the operation detection section 183.

Thus, the pointing information including the operation position of theoperation performed by the commander in the HMD 100A, the positions ofthe menu image 411 and so on during the operation, and the data fordesignating the menu operation is transmitted from the HMD 100A to theHMD 100B. The display position calculation section 321 of the HMD 100Bdisplays the menu image 471 as shown in FIG. 10B in accordance with thepositions of the menu image 411 and so on. It should be noted that it ispreferable that the positions of the operation sections such as iconsprovided to the menu image 471 are the same or similar to those of themenu image 411 in the HMD 100A. Therefore, it is also possible toinclude the data for designating the type or the configuration of themenu image 411 in the pointing information generated and thentransmitted by the HMD 100A. The display position calculation section321 displays the image 453 of the pointing object based on the pointinginformation. In the case in which the display position of the image 453of the pointing object overlaps the display positions of the operatingsections such as icons of the menu image 471, the pointing detectionsection 319 detects the operation to the operation section.

Therefore, when the commander performs the position pointing operationwith the pointing object 2 in the HMD 100A, the HMD 100B detects theoperation in accordance with the position pointing operation, and thus,it is possible to make the HMD 100B perform the action. Therefore, itbecomes possible for the commander to remotely operate the HMD 100B.

Further, in the case in which the display system 1 is configuredincluding the plurality of HMD 100B, the HMD 100A receives the workinformation from the plurality of HMD 100B and then displays the workinformation.

FIG. 10C shows an example in which the HMD 100A displays the takenimages based on the work information received from the two HMD 100B.Among the display areas 409, 410 disposed in the display area 401, inthe display area (here, the display area 409) on the target side of theoperation using the menu image, there is displayed the menu image 413.When the operation using the pointing object 2 is performed inaccordance with the position of the menu image 413, the pointinginformation including the operation position and the positions of themenu image 413 and so on is transmitted to the HMD 100B corresponding tothe display area 409. Further, during the operation to the display area409, the display based on the work information of another HMD 100Bcontinues in the display area 401.

Further, in the case in which the display system 1 includes a pluralityof HMD 100B, it is also possible to transmit the pointing information,which has been transmitted by the HMD 100A to one of the HMD 100B, toanother of the HMD 100B. FIG. 10D shows the display example in thiscase. As shown in FIG. 10D, the content displayed by the HMD 100B in thedisplay area 451 as shown in, for example, FIG. 10B can be displayed inanother HMD 100B. The work object is not necessarily included in theoutside view of the HMD 100B which displays the display area 461.Therefore, in the display area 461, the image 462 of the work objecttaken by the HMD 100B is displayed together with the image 463 of thepointing object, and the menu image 473. Further, since the HMD 100B fordisplaying the display area 461 is not the target of the operation bythe HMD 100A, the HMD 100B is not required to perform the actioncorresponding to the operation of the pointing object 2. Further, it isalso possible for the HMD 100B to perform the action corresponding tothe pointing object 2 similarly to other HMD 100B. In this case, it ispossible for the commander wearing the HMD 100A to operate the pluralityof HMD 100B.

As described above, the control section 300 detects the operation to theHMD 100B based on the information representing the operation included inthe pointing information received in the communication section 117. Morespecifically, the control section 300 makes the image display section 20display the menu image 471 which is an image for the GUI operation. Thepointing detection section 319 detects the operation based on thedisplay position of the menu image 471 and the operation positionincluded in the pointing information received by the communicationsection 117. Therefore, by performing the position pointing operation inthe HMD 100A, it is possible to perform the GUI operation to the HMD100B to control the HMD 100B.

Third Embodiment

FIGS. 11A and 11B are flowcharts showing an action of the display system1 according to a third embodiment. FIG. 11A shows an action of the HMD100A, and FIG. 11B shows an action of the HMD 100B.

Since the display system 1 according to the third embodiment isconfigured similarly to the display system 1 explained in the abovedescription of the first embodiment, graphical description and theexplanation of the constituents of the display system 1 will be omitted.Further, in the flowcharts shown in FIGS. 11A and 11B, the actionscommon to the first embodiment and the third embodiment and alreadyexplained in the description of the first embodiment (FIGS. 6A and 6B)are denoted with the same step numbers.

In the third embodiment, the point that the work information includingthe taken image taken by the camera 61 of the HMD 100B is transmitted tothe HMD 100A, and is displayed in the HMD 100A is substantially the sameas in the first embodiment. In the third embodiment, it is possible toedit the taken image taken by the HMD 100B using the position pointingoperation of the HMD 100A, and display the taken image thus edited inthe HMD 100B.

In the actions of the HMD 100A shown in FIG. 11A, the steps S11 throughS15 are as explained in the above description of the first embodiment.In the display system 1 according to the third embodiment, an operationcorresponding to the content is performed in the step S14, and afterthis operation is detected by the operation detection section 183, anoperation to the taken image presently displayed can further beperformed (step S51). The operation detection section 183 detects (stepS52) the operation to the taken image, and then makes the operationdetected in the steps S15 and S52 correspond to the taken image (stepS53). Specifically, the operation detection section 183 generates thedata representing the operation position detected in the step S15 as arelative position to the taken image. Further, the operation detectionsection 183 generates the data of the taken image edited using theoperation detected in the step S52.

Subsequently, the transmission image acquisition section 185 obtains(step S54) the transmission image, and the pointing informationgeneration section 186 generates and then transmits (step S55) thepointing information including the image data of the image of thepointing object obtained by the transmission image acquisition section185, and the data generated by the operation detection section 183. Theprocess of the transmission image acquisition section 185 obtaining thetransmission image is a process substantially the same as, for example,the process in the step S17.

Further, the actions of the HMD 100B shown in FIG. 11B are as explainedin the above description of the first embodiment. Specifically, the HMD100B receives (step S24) the pointing information transmitted by the HMD100A, and then the received information analysis section 317 analyzesthe pointing information thus received to extract (step S25) the imagedata. In the step S25, the taken image data of the HMD 100B edited inthe HMD 100A, and the image data of the pointing object are extracted.Subsequently, the display position calculation section 321 calculates(step S26) the position where the image is displayed, and the displayinformation generation section 325 generates the display information tomake the image processing section 303 and the display control section305 display (step S27) the image.

According to the display system 1 related to the third embodiment, inthe case in which the work information received from the HMD 100Bincludes the taken image, the control section 140 makes the imagedisplay section 20 display the taken image, and then edits the takenimage based on the operation detected by the operation detection section183 during the period of displaying the taken image. Further, thecontrol section 140 generates the pointing information including thetaken image thus edited, and then transmits the result to the HMD 100B.The HMD 100B receives the pointing information including the taken imagethus edited, and then display the taken image thus edited using theimage display section 20.

Thus, it is possible to show the result, which is obtained by thecommander virtually performing the operation to the work object locatedin the view field of the HMD 100B, to the worker. Therefore, regardingthe work to the work object, a larger amount of information can beprovided to the worker.

Further, although in the third embodiment, there is explained an exampleof editing the taken image taken by the HMD 100B using the HMD 100A, itis also possible to, for example, transmit document data or drawing datafrom the HMD 100B to the HMD 100A, and then edit the document data andthe drawing data using the HMD 100A. In this case, the document data andthe drawing data thus edited are transmitted to the HMD 100B, and theHMD 100B performs the display based on the document data and the drawingdata thus edited. Thus, regarding the work of editing the document,generating and editing the drawing, it is possible for the commander toperform editing to thereby show a so-called example to the worker.

Fourth Embodiment

FIGS. 12A and 12B are flowcharts showing an action of the display system1 according to a fourth embodiment. FIG. 12A shows an action of the HMD100A, and FIG. 12B shows an action of the HMD 100B.

Since the display system 1 according to the fourth embodiment isconfigured similarly to the display system 1 explained in the abovedescription of the first embodiment, graphical description and theexplanation of the constituents of the display system 1 will be omitted.Further, in the flowcharts shown in FIGS. 12A and 12B, the actionscommon to the first embodiment and already explained in the descriptionof the first embodiment (FIGS. 6A and 6B) are denoted with the same stepnumbers.

In the fourth embodiment, it is arranged that the direction to be imagedby the camera 61 of the HMD 100B can be controlled by the HMD 100A.Therefore, in accordance with the instruction of the commander wearingthe HMD 100A, the imaging direction of the camera 61 of the HMD 100B canbe controlled. If necessary, in order to change and adjust the imagingdirection of the camera 61, it is possible to dispose a ball joint or arotating mechanism capable of changing the direction of the camera 61 inthe pedestal of the camera 61, and to provide a manipulator or the likefor moving the camera 61. It is sufficient for the manipulator to actunder the control of the control section 300, and it is also possible toadopt a configuration in which the imaging control section 313 performsthe control.

In the actions of the HMD 100A shown in FIG. 12A, the steps S11 throughS12 are as explained in the above description of the first embodiment.

In the display system 1 according to the fourth embodiment, afterdisplaying the taken image in the HMD 100A, the operation of instructingthe imaging direction can be made (step S71). The control section 140 ofthe HMD 100A generates the information of instructing the imagingdirection based on the operation detected by the operation detectionsection 183, and then transmits (step S72) the information using thecommunication section 117.

In the actions of the HMD 100B shown in FIG. 12B, the steps S21 throughS23 are as explained in the above description of the first embodiment.

The HMD 100B receives (step S81) the instruction transmitted by the HMD100A, and then the operation detection section 183 updates (step S82)the imaging direction of the camera 61 in accordance with theinstruction thus received. Subsequently, the operation detection section183 starts (step S82) a stabilizing process. The stabilizing process isa process of stabilizing the taken image in the case in which it is noteasy to stabilize the imaging direction of the camera 61 since theposition and the direction of the HMD 100B are changed due to themovement of the worker. Specifically, in the case in which the camera 61is provided with an optical image stabilization function of shifting theimaging element or the lens, the operation detection section 183corrects the tremor of the taken image using the image stabilizationfunction of the camera 61. Further, it is also possible for theoperation detection section 183 to perform an image processing forcorrecting the tremor on the taken image data of the camera 61.

In the HMD 100A, the commander can perform an operation of instructingthe capture of the taken image of the HMD 100B. Specifically, althoughit is possible to adopt a configuration of always transmitting the takenimage data taken by the HMD 100B from the HMD 100B to the HMD 100A, inorder to reduce the power consumption and the amount of datacommunication, the taken image data is transmitted in accordance withthe instruction of the worker in the HMD 100B. In the presentembodiment, it is possible for the commander to operate the HMD 100A toinstruct the transmission of the image. The control section 140 detectsthe operation of the commander to generate the instruction for capture,and then transmits (step S73) the instruction to the HMD 100B.

In the HMD 100B, the control section 300 receives (step S84) theinstruction of the capture, and then makes the camera 61 perform theimaging (step S85). In the case in which the camera 61 has theconfiguration of always performing imaging, the control section 300obtains the taken image data in the step S85. The control section 300generates the work information including the taken image data thusobtained, and then transmits (step S86) the work information to the HMD100A. The work information transmitted in the step S86 can be the sameinformation as the information transmitted in the step S23, or the datahaving a different configuration providing the taken image data isincluded.

The HMD 100A receives and then analyzes (step S74) the work informationtransmitted from the HMD 100B, and then displays (step S75) the takenimage taken in the HMD 100B using the image display section 20.

According to the display system 1 related to the fourth embodiment, theimaging condition with which the camera 61 of the HMD 100B takes theimage is designated by the HMD 100A, and the HMD 100B controls thecamera 61 based on the imaging condition thus designated. Therefore, inthe case in which the command site A and working site B are distant fromeach other, it is possible for the commander using the HMD 100A toobserve the desired direction and the desired place. The imagingcondition includes, for example, the imaging direction of the HMD 100B.The imaging direction can be defined as the direction to which thecenter axis of the imaging range (field angle) of the camera 61 faces.The taken image data transmitted by the HMD 100B to the HMD 100A is thetaken image not following the viewpoint of the worker, and the imagingdirection can be controlled by the instruction of the commander.Therefore, it is possible for the commander to obtain the desiredinformation with respect to the working site B as the environment of theHMD 100B, and to efficiently perform the work support, instructions, andso on. Further, since the control of actually changing the imagingdirection is performed by the control section 300 of the HMD 100B, anincrease in the load of the HMD 100A and an increase in the amount ofthe communication data can be avoided. Further, since the stabilizingprocess is performed under the control of the control section 300, theimage, which does not follow the motion of the worker, and isstabilized, can be observed in the HMD 100A.

In the fourth embodiment, the HMD 100B is not limited to the case oftransmitting the taken image itself of the camera 61, but can alsotransmit the information generated from the taken image to the HMD 100A.In this case, it is sufficient for the HMD 100A to display theinformation generated by the HMD 100B based on the taken image, or tooutput the information as a sound instead of the process of displayingthe taken image. Further, in the fourth embodiment, it is possible todesignate the zoom factor as the imaging condition. Specifically, it ispossible for the HMD 100A to transmit the instruction of designating thezoom factor in the camera 61 of the HMD 100B in addition to the imagingdirection. By controlling the zoom factor, it is possible to adjust notonly the magnification in performing imaging in the HMD 100B, but alsothe imaging range. Further, as the method of designating the imagingdirection by the HMD 100A, it is also possible to adopt a method of, forexample, designating the imaging direction so that a specific objectshowing the taken image of the camera 61 of the HMD 100B is fit into thetaken image. In this case, it is sufficient for the control section 300to analyze the taken image data to control the imaging direction and thezoom factor so that the specific object fits into the imaging range.Further, the information transmitted by the HMD 100A and for designatingthe imaging condition such as the information transmitted in the stepS72 can also be included in the pointing information. In this case, itis possible for the control section 300 to perform the action shown inFIG. 12B in the series of processes for receiving and analyzing thepointing information.

In the fourth embodiment, it is possible to adopt a configuration inwhich the control section 300 of the HMD 100B clips a part of theimaging range (field angle) of the camera 61 to transmit the part to theHMD 100A. In this case, the control section 300 updates the clippingrange of the taken image of the camera 61 in the case of changing theimaging direction under the control of the HMD 100A. For example, if thecameras 61 of the HMD 100B, 100C are each constituted by a wide-anglelens, such an embodiment can easily be realized. It should be noted thatthe invention is not limited to the configuration of each of theembodiments described above, but can be implemented in various formswithin the scope or the spirit of the invention.

For example, it is possible to adopt an image display section of adifferent type such as an image display section to be worn like a hatinstead of the image display section 20, and it is sufficient to beprovided with a display section for displaying an image so as tocorrespond to the left eye of the user, and a display section fordisplaying an image so as to correspond to the right eye of the user.Further, it is also possible for the display device according to theinvention to adopt a configuration as the head mounted display installedin a mobile object such as a vehicle or a plane. Further, it is alsopossible to adopt a configuration as the head mounted displayincorporated in a body protector such as a helmet. In this case, a partfor positioning the device with respect to the body of the user and apart positioned with respect to the part can be used as the mountingsections.

Further, in the above description of the embodiments, the explanation ispresented assuming that there is adopted the configuration in which theHMD 100A, 100B each have the image display section 20 and the controldevice 10A, 10B separated from each other and connected via theconnection section 40. The invention is not limited to thisconfiguration, but it is also possible to adopt a configuration in whichthe control device 10A, 10B and the image display section 20 areintegrally configured, and are mounted of the head of the user.

Further, it is sufficient for the HMD 100A, 100B to be a device mountedon the head of the user (the worker, the commander) in at least theimage display section 20 for performing the display, and the mountingstate of the control device 10A, 10B is not particularly limited.Therefore, a laptop computer, a tablet computer, or a desktop computercan also be used as the control device 10A, 10B. Further, as the controldevice 10A, 10B, a portable electronic apparatus including a gamemachine, a cellular phone, a smart phone, and a portable media player,and other dedicated equipment can be used as the control device 10A,10B. Further, it is also possible to adopt a configuration in which thecontrol device 10A, 10B is configured separately from the image displaysection 20, and the variety of signals are wirelessly transmitted andreceived between the control device 10A, 10B and the image displaysection 20.

Further, for example, as the configuration of generating the image lightin the image display section 20, it is also possible to adopt aconfiguration including an organic EL (organic electroluminescence)display and an organic EL control section. Further, as the configurationof generating the image light, a LCOS (Liquid Crystal On Silicon; LCoSis a registered trademark), a digital micromirror device, and so on canbe used. Further, as the optical system for guiding the image light tothe eyes of the user, there can be adopted a configuration in which anoptical member for transmitting the outside light entering from theoutside toward the device, and the outside light is made to enter theeyes of the user together with the image light. Further, it is alsopossible to use an optical member located in front of the eyes of theuser to overlap a part or the whole of the field of view of the user.Further, it is also possible to adopt a scanning type optical system forcausing a scanning movement of the laser beam or the like to form theimage light. Further, the invention is not limited to those guiding theimage light inside the optical member, but the configuration onlyprovided with a function of guiding the image light by refracting and/orreflecting the image light toward the eyes of the user can also beadopted.

For example, it is also possible to apply the invention to a laserretinal projection head mounted display. Specifically, it is alsopossible to adopt a configuration in which the light emitting section isprovided with a laser source and an optical system for guiding the lasersource to the eyes of the user, makes the laser beam enter the eyes ofthe user, and then scans the surface of the retina with the laser beamto provide an image on the retina to thereby make the user visuallyrecognize the image.

Further, it is also possible to apply the invention to a display deviceadopting the scanning optical system using an MEMS mirror, and using theMEMS display technology. In other words, as the light emitting section,it is also possible to provide a signal light forming section, ascanning optical system having the MEMS mirror for causing the scanningmovement of the light emitted by the signal light forming section, andan optical member on which a virtual image is formed by the light thescanning movement of which is caused by the scanning optical system. Inthis configuration, the light emitted by the signal light formingsection is reflected by the MEMS mirror, enters the optical member, andis guided through the optical member, and then reaches a virtual imageforming surface. By the MEMS mirror causing the scanning movement of thelight, the virtual image is formed on the virtual image forming surface,and by the user figuring out the virtual image with the eyes, the imageis recognized. An optical component used in this case can be a componentfor guiding the light through a plurality of times of reflectionsimilarly to, for example, the right light guide plate 261 and the leftlight guide plate 262 in the embodiment described above, or can berealized using a half-mirror surface.

Further, the optical elements according to the invention is not limitedto the right light guide plate 261 and left light guide plate 262respectively including the half mirrors 261A, 262A, but can sufficientlybe the optical components for making the image light enter the eyes ofthe user, and specifically, a diffraction grating, a prism, and aholography display section can also be used.

Further, at least a part of the functional blocks shown in FIGS. 4 and 5can be realized with hardware, or can be provided with a configurationrealized by cooperation between hardware and software, and the inventionis not limited to the configuration of arranging the independenthardware resources in the same manner as shown in FIGS. 4 and 5.Further, the programs executed by the control section 140 can also bestored in the storage section 120 or the storage device in the controldevice 10A, or stored in the ROM provided to the control section 140.The programs executed by the control section 300 can also be stored inthe storage section 340 or the storage device in the control device 10B,or stored in the ROM provided to the control section 300. Further, theprograms executed by the control section 140 can be stored in a storagedevice or a storage medium externally connected to the HMD 100A. Theprograms executed by the control section 300 can similarly be stored ina storage device or a storage medium externally connected to the HMD100B. Alternatively, it is also possible to install the programs fromthese storage device and the storage medium into the storage sections120, 340 and then execute the programs. Further, it is also possible toadopt a configuration of obtaining the programs stored in the externaldevice via the communication section 117 or the interface 114, and thenexecuting the programs. Further, the constituents formed in the controldevice 10A, 10B can also be formed in the image display section 20 in anoverlapping manner. For example, it is also possible to adopt aconfiguration in which the control section 140 shown in FIG. 4 and thecontrol section 300 shown in FIG. 5 are also disposed in the imagedisplay section 20, and in this case, it is possible for the controlsection in the control device 10A, 10B and the control section of theimage display section 20 to share the function.

The entire disclosure of Japanese Patent Application No. 2014-263853,filed Dec. 26, 2014 is expressly incorporated by reference herein.

What is claimed is:
 1. A display system comprising: a first displaydevice of a head mounted type, and a second display device, wherein thefirst display device includes: a first display configured to display aguide image, a first camera configured to capture an outside image, afirst communication interface configured to: transmit the outside imageto the second display device, and receive the guide image transmittedfrom the second display device, and the second display device includes:a second display configured to display the outside image, a detectorconfigured to detect a pointing position relative to the outside image,a second processor configured to generate the guide image based on thepointing position, a second communication interface configured to:receive the outside image transmitted from the first display device, andtransmit the guide image to the first display device.
 2. The displaysystem according to claim 1, wherein the first display device includes afirst processor configured to recognize a work object within the outsideimage, and cause the first display to display an image of a pointingobject at the pointing position relative to the work object.
 3. Thedisplay system according to claim 1, wherein the guide image includes anarrow.
 4. The display system according to claim 1, wherein the guideimage includes an indication of a motion.
 5. The display systemaccording to claim 1, wherein the guide image includes an indication ofa trajectory of an operation to be performed.
 6. The display systemaccording to claim 1, wherein the guide image includes an image of atleast a part of a body.
 7. The display system according to claim 1,wherein the guide image is displayed with a fluorescent color.
 8. Thedisplay system according to claim 1, wherein the guide image includes amenu of icons.
 9. A head mounted display device comprising: a firstdisplay configured to display an image while allowing an outside view tobe visually recognized; a camera configured to capture an image of arange including at least a part of the outside view; a communicationinterface configured to: transmit work information including a capturedimage to a second display device, and receive guide informationtransmitted from the second display device, and a first processorconfigured to: cause the display to display imagery based on thereceived guide information, wherein the second display device isconfigured to, upon receipt of the work information including thecaptured image by the first display device: control the second displaydevice to display the captured image, detect a position pointed to by apointing object, calculate a pointing position that is the positionpointed to by the pointing object relative to the captured imagepresently displayed by the second display device, generate the guideinformation that includes the calculated pointing position pointed to bythe pointing object relative to the captured image, and transmit thegenerated guide information to the first display device.
 10. Aninformation display method comprising: capturing an outside image with acamera of a first display device of a head mounted type; transmitting,by the first display device, the outside image captured by the camera ofthe first display device to a second display device by a firstcommunication interface of the first display device; causing a displayof the second display device to display the outside image received fromthe first display device by a second communication interface of thesecond display device; detecting a pointing position pointed to relativeto the outside image displayed by the display of the second displaydevice; generating, by a processor of the second display device, a guideimage based on the pointing position; transmitting, by the secondcommunication interface of the second display device, the guide image tothe first display device; receiving, by the first communicationinterface of the first display device, the guide image transmitted fromthe second display device; and displaying, on a display of the firstdisplay device, the guide image received from the second display device.11. A non-transitory computer readable recording medium comprisingcomputer program instructions that, when executed by a processor, causethe processor to: cause a display of a first display device to displayan image while allowing an outside view to be visually recognized, thefirst display device being of a head mounted type; transmit workinformation including an image captured by a camera of the first displaydevice to a second display device; receive guide information transmittedfrom the second display device; and cause the display of the firstdisplay device to display imagery based on the received guideinformation, wherein the second display device is configured to: uponreceipt of the work information including the captured image by thesecond display device, control the second display device to display thecaptured image, detect a position pointed to by a pointing object,calculate a pointing position that is a position pointed to by thepointing object relative to the captured image presently displayed bythe second display device, generate the guide information that includesthe calculated pointing position pointed to by the pointing object, andtransmit the generated guide information to the first display device.